Reliability and validity of the Hungarian version of the Personality Inventory for DSM-5 (PID-5)

Objectives and methods In order to assess the internal consistency, fit indexes, test-retest reliability, and validity of the Personality Inventory for the DSM-5 (PID-5) and its associations with age, gender, and education, 471 non-clinical (69,6% female; mean age: 37,63) and 314 clinical participants (69,7% female, mean age: 37,41) were administered the Hungarian translation of the PID-5, as well as the SCL-90-R and the SCID-II Personality Questionnaire. Results We found that; (a) temporal consistency of the Hungarian PID-5 was confirmed by one-month test-retest reliability analysis, (b) validity of the PID-5 instrument is acceptable in the clinical and the non-clinical sample as well, based on significant correlations with SCID-II and SCL-90-R, (c) PID-5 facets’ and domains’ associations with gender, age, and level of education are in accordance with previous findings. Conclusion These findings support that the Hungarian PID-5 is a reliable and valid instrument for both clinical and non-clinical populations.


Introduction
In the Alternative Model for Personality Disorders (AMPD) in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5, section III), specific personality disorders (PD) are defined by typical impairments in personality functioning (Criterion A) and characteristic pathological personality traits (Criterion B) [1]. The AMDP retained only 6 of the 10 DSM-IV PDs and changed the Personality Disorder Not Otherwise Specified (PDNOS) category to a PD-Trait Specified diagnosis. The AMDP has not been accepted as the official diagnostic system for use in the DSM-5, partly because of the limited research support for its specific components. Since its publication, more than hundred publications investigated the validity of the AMDP in different languages [3,7] and it needs further validation and examination whether it is generalizable to Hungarian language and culture. Although another study has already been published about the psychometric properties of the PID-5 in a Hungarian clinical and community sample [19], we found that this study had some psychometric and scientific limitations besides its strengths, and it is worth using another adaptation of the PID-5 mesure.
First of all, our Hungarian translation of the PID-5 scale had been approved by the American Psychiatric Association (APA). Moreover, we added some important extensions to the investigation of the psychometric properties of the PID-5 by including a 1-month test-retest reliability of the scales, by using SCID-II and SCL-90-R correlations as validity criteria, and by presenting three different models of the PID-5 internal structure in the confirmatory factor analysis.
Pathological personality traits (Criterion B), proposed by Krueger, Derringer, Markon, Watson, and Skodol [2] are organized into five broad domains: Negative Affectivity, Detachment, Antagonism, Disinhibition, and Psychoticism. Within the five broad trait domains, there are 25 specific trait facets [1]. APA (https://www.psychiatry.org/psychiatrists/practice/ dsm/educational-resources/assessment-measures) provided different kinds of measures (selfreport, informant, rating scale) for assessing maladaptive personality traits. The Personality Inventory is a self-report and other-informant measure for the DSM-5 (PID-5) [2]. Since its introduction, several studies examined various aspects of PID-5's psychometric properties in non-clinical and clinical samples (see a review by Al-Dajani et al. [3]). We aim to further examine the psychometric properties of the Hungarian version of PID-5 in non-clinical and clinical samples. Our decision to use a non-clinical sample as well is based on the theory that traits that describe personality disorders are experienced at a less extreme level among healthy persons [4].
Internal consistencies of PID-5 domains and facets have been studied extensively, and mixed internal consistency scores were demonstrated. Coefficient alphas [5] for domain scales were between .75-.92 and .87-.96 and for facets ranged from .46-.77 to .94-.96, demonstrating some possible issues with scale reliability at the facet level (see Al-Dajani et al. [3]). Two versions of the domain scoring algorithm are used in the literature. The 25 facets version, which uses all the 25 facets in the definitions of the five domains, and the 15 facets version, which uses only 3 facets per domain [6]. Both versions derived from the study of Krueger et al. [2] which identified five higher order factors by explorative factor analysis of 25 facets. In order to eliminate cross-loadings of facets, three traits with the highest factor loadings were selected for each domain [2]. This scoring instruction was published in the APA copyright version of the PID-5 [6]. Further studies that investigated the five-factor exploratory factor model of the PID-5 facets also found facets that have cross-loadings on more than one domain and the number, magnitude and domain location of the loadings for crossloading facets varies across study samples [7].
In an attempt to address the variable nature of PID-5 factor analytic results, Watters and Bagby [7] conducted a meta-analysis of the PID-5 lower-order factor structure. They found that the level of crossloadings decreased when multiple samples were combined, and that the 15 facets scoring version of domains showed the lowest number of cross-loading facets (only anhedonia loaded on two domains substantially). A comparison of the factor structures of the three models is in Table 1.
One of our objective is to compare the factor structure of the 15 and 25 facets scoring version by confirmatory factor analysis. We also intend to quantify whether substantive differences in results occur because of using different domain scoring methods. In order to compare the different methods used in research examining the psychometric properties of PID-5 with exploratory or confirmatory factor analysis, Table 2 presents an overview of methods for such studies. This mini-review is also a means to compare the results of studies that analyze the psychometric properties of PID-5 in different languages.
We found one study that examined the test-retest reliability of the PID-5. Wright et al. [8] found that across an average of 1.44 years, Cohen's d ranged between .17 (Antagonism) and 0.00 (Negative Affectivity) for the domains whose range is in the little to no change area according to Cohen's guidelines [9] and between little and small change was found for the facets (d = .02 (Depressivity) and -.30 (Submissiveness).
The current study aimed to examine the psychometric properties of the Hungarian PID-5 facet and domain scales among Hungarian-speaking adults from a non-clinical community sample as well as in a Hungarian clinical sample. Specifically, we analyzed its facets and domains' internal consistency and test-retest reliability. In addition, we tested the assumption of whether the DSM-5 traits are indeed maladaptive personality traits by analyzing their correlations with general personality disorder severity measured by the sum of the SCID-II screening questionnaire and general distress measured by SCL-90-R GSI. We also analyzed the specific associations between PID-5 domains and facets with personality disorder dimensions measured by the SCID-II screening questionnaire and the nine symptom dimensions of SCL-       -Exploratory factor analysis replicated the initially reported five-factor structure as indicated by high factor congruencies. The two-, three-, and four-factor solutions estimated in the hierarchy of the DSM-5 traits bear close resemblance to existing models of common mental disorders, temperament, and personality pathology. Thus, beyond the description of individual differences in personality disorder, the trait dimensions might provide a framework for the metastructure of psychopathology in the DSM-5 and the integration of a number of ostensibly competing models of personality trait covariation.
Participants completed all questionnaires online in Google-docs, a secure online survey tool. Each participant gave written consent to participate by choosing the "agree with" option in response to an informed consent form.
In the clinical sample some of the data was missing, and these missing data were not replaced, therefore the sample sizes vary in each analysis. There were 92 individuals who did not mark an answer for at least one item of the PID-5 questionnaire. One participant was excluded from the analysis due to too much missing data (50%), for the others, the number of missing data did not exceed 10%. The sample size was the lowest (222) in the analyzes where all PID-5 items were included in the confirmatory factor analysis (CFA). In CFAs when the complete factor structure was analyzed, the sample sizes varied between 222-241. Sample sizes ranged from 295 to 307 for item-level analyzes and from 265 to 277 for domain-level analyzes. No data were missing in the analyzes on facet-and domain scales, as the scores on the scales were calculated by averaging.
In addition, 93 (62 female) non-clinical subjects participated in the test-retest reliability study. The age of the participants in the test-retest reliability study varied from 19 to 62 years (Mean: 40.32; SD: 13.08). The descriptive statistics of the samples can be found in Table 3. The local Ethical Committee of the ELTE PPK (2018/8) approved the study protocol. The Regional and Institutional Committee of Science and Research Ethics of Semmelweis University approved the research procedure.

Personality Inventory for the DSM-5 (PID-5).
The PID-5 is a 220-item inventory that assesses personality disorder traits according to the DSM-5 trait model [2]. Items are rated on a 4-point Likert-type scale from 0 (Very False or Often False) to 3 (Very True or Often True). The PID-5 yields scale scores for 25 facets and the following five domains: Negative Affectivity, Detachment, Antagonism, Disinhibition, and Psychoticism. The official Hungarian translation used in the current study has been permitted by APA. Two independent translators translated the PID-5 into Hungarian, then a group of experts compared the two translations and agreed on a first version. We distributed the first version among patients of the Department of Psychotherapy at Semmelweis University, who commented on it, and based on their comments, we made corrections to the translation. There was no back translation made. A Hungarian version is freely available at the following site: https://lelekbenotthon. hu/pid-5-dsm-5-szemelyisegkerdoiv/ [10,11] contains 119 items in a yes/no response format, corresponding to the DSM-IV personality disorders' criteria. Regional and Institutional Committee of Science and Research Ethics of Semmelweis University recommended to leave out items referring to antisocial personality disorder, therefore we used only 104 items. In the present study, the PDs were expressed dimensionally by averaging the scores (range 0-1) of each SCI-D-II-PQ item per PD. Higher scores are reflecting more of the personality characteristic being assessed. We followed Hopwood et al.'s [12] method and eschewed categorical PDs in favor of continuous symptom counts for this study. In our non-clinical sample, where participants do not fit to a full PDs diagnosis, the use of continuous scales of PDs seemed to be a viable method. The SCID-II screening questionnaire has good reliability and validity data [13]. For the current study, we calculated PD scores for those PD-s which are proposed to be retained in DSM-5-except for Antisocial Personality Disorder -, as well as a total score of SCID-II by averaging. We did not measure the symptoms of Antisocial Personality Disorder since our ethical committee did not allow us to use the antisocial modul of the SCID-II-PQ for legal and ethical reasons. The internal consistency of SCID-II-PQ scales ranged from α = .55 (schizoid) to α = .89 (borderline). -90-R). In order to assess the level of general distress and nine specific symptom dimensions (Somatization (SOM); Obsessive-compulsive (O-C); Interpersonal sensitivity (I-S); Depression (DEP); Anxiety (ANX); Hostility (HOS); Phobic anxiety (PHOB); Paranoid ideation (PAR); and Psychoticism (PSY), the Hungarian version of the SCL-90-R [14] was used. For analysis, a Global Severity Index (GSI) and the subscale scores were calculated by averaging. The internal consistency of these scales ranged from α = .82 (anger-hostility) to α = .95 (depression).

Data analytic procedure
MANOVA was run to analyze gender differences along with the 25 personality disorder traits, Pearson and Spearman's correlation were calculated to evaluate the relationship between PID5 traits and age as well as educational level. The reliability of the Hungarian PID-5 was examined by calculating the internal consistencies of the facet and domain scores (Cronbach's alpha). We examined the 1-month test-retest reliabilities for the PID-5 scales by using Pearson correlation coefficients and Cohen's d, which can establish a magnitude of change within this time interval. According to Cohen [9] d of .2 -.5 means small change, d of .5 -.8 means medium change, and if the value of d is higher than 0.8 the change can be considered large.
Confirmatory factor analysis (CFA) was conducted to test the unidimensionality of the PID-5 scales and to explore the factor structure of PID-5 facets and domains. Model fit was evaluated using the usual fit indexes: Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), Standardized Root Mean Square Residual (SRMR). The CFI is acceptable if the value is higher than .90, the RMSEA and SRMR indexes should be less than .10 for adequate fit [15][16][17].
To examine the validity of the Hungarian PID-5, relationships of PID-5 scales with scales of SCID II PQ and SCL90-R were explored using correlation analysis. In summary, most scales have appropriate reliability, except for the Suspiciousness facet in the non-clinical sample, the reliability of which is low.

Test-retest reliability and magnitudes of change.
As it is shown in Table 4, internal reliability coefficients of PID-5 dimensions on test-retest sample were between .64 for Irresponsibility, .96 for Eccentricity. Test-retest correlation coefficients (Pearson) were between .53 (Callousness) and .86 (Risk-taking), which reflects a moderate and strong connection between the two measurements. Utilizing Cohen's d to establish magnitude of change within a one month interval, a small amount of change (i.e., d of . In other domains and facets, we found a smaller d than-.20, which reflects little to no change according to Cohen's guidelines (Cohen, 1988).

Confirmatory factor analysis.
We ran a Confirmatory Factor Analysis in order to explore the factor structure of PID-5 traits; results are reported in Tables 4 and 5. We analyzed the fitting of all the three factor structures found in the literature (the APA model [1], the early Krueger model [2] and the Watters-Bagby model [7]). The results showed that the RMSE and SRMR fit indices reach the good or acceptable level in both samples in case of all the three models, while CFI scores indicate a poor fit in all samples and models in almost all domains.

Correlations between PID-5 traits and SCID-II as well as SCL-90-R scores.
Correlations between PID-5 traits and SCID-II personality disorder scores and SCL-90-R scores are reported in Tables 6 and 7. We found that correlations of PID-5 domains and SCID-II total PD score are significant in both the clinical and the non-clinical groups with a Bonferroni adjusted p-value. Effect sizes are moderate or strong. Correlations between the traits and SCI-D-II total score are stronger in the clinical sample, where correlations are moderate to strong, except for weak correlations with Intimacy Avoidance, Submissiveness and Risk Taking. In the non-clinical sample, correlations between the traits and SCID-II total score are mostly moderate, with the exceptions of avoidance, Submissiveness, Risk Taking and Rigid Perfectionism, Restricted Affectivity and Attention Seeking, which are also weakly correlated.
Different personality disorder scores of SCID-II-PQ are significantly correlated to the expected PID-5 domains and traits with moderate or strong effect size, however, the correlations are stronger in the clinical sample. There are only a few exceptions: Intimacy Avoidance   has only a weak relationship with Avoidant-and Obsessive-Compulsive PD. Restricted Affectivity is weakly correlated to Schizotypal-and Obsessive-Compulsive PD and the correlation between Borderline PD and Risk Taking is also weak. Total score of SCL-90-R has significant (Bonferroni corrected) correlations with all PID-5 scales, except for Risk Taking in the clinical sample. Correlations are moderate to strong, except for Attention Seeking, Callousness, Grandiosity, Impulsivity, Irresponsibility, Restricted Affectivity in both samples as well as Deceitfulness and Submissiveness in clinical sample. The effect sizes of these latter correlations are weak.
At domain level, the PID-5 negative affectivity, detachment, disinhibition and psychoticism show significant, moderate or strong correlations with all the SCL90-R scales. PID-5 antagonism has weak relationship to SCL-90 R scales. With regards to the facet level, as expected, the strongest relations were found between the PID-5 and the SCL-90-R counterparts. PID-5 anxiousness shows moderate correlation with the SCL-90-R anxiety scale and the PID-5 depressivity displays high correlation with the SCL-90-R depression scale. Furthermore, PID-5 hostility is correlated with SCL-90-R hostility, while PID-5 suspiciousness correlates with the SCL-90-R paranoid ideation, all of them with moderate effect sizes.
However, there are several PID-5 traits that do not correlate with some of the SCL-90-R dimensions in the clinical sample, these are: callousness, deceitfulness, grandiosity, intimacy avoidance, manipulativeness, risk taking and submissiveness.

Differences between clinical and non-clinical samples, gender, age and education.
Associations between PID-5 domains and facets comparing clinical and non-clinical groups, gender differences, as well as correlations with age and education are reported in Table 8.

Clinical and non-clinical differences.
MANOVA was run to analyze clinical and non-clinical differences along with the 25 personality disorder traits and the 5 domains. Based on the 25 traits we found significant differences with a large effect size (Wilks's l = .449; F (25, 759) = 37.31; p < .001; h 2 = .551). More specifically, the Manipulativness, Deceitfulness and Risk Taking traits did not show significant differences between the clinical and non-clinical groups; for all the other traits, scores were significantly higher in the clinical group. At the domain level, the difference was also significant with a large effect size (Wilks's l = .564; F (25, 779) = 120.606; p < .001; h 2 = .436).
3.2.2.2. Gender differences. According to our results, the male and female profiles of personality disorder traits were significantly different and the difference had a large effect size in both the clinical (Wilks's l = .755; F (25, 288) = 3.735; p < .001; h 2 = .245) and the non-clinical   We also found some significant but very weak correlations with education on both the domain and the facet levels. There is a weak negative correlation between Suspiciousness and education in the clinical sample (r = 28).
Based on our results our conclusion is that we recommend the Hungarian version of the PID-5 for future clinical research and practice. In the following, we highlight selected findings that contribute to the current literature on the PID-5 and the new DSM-5 trait model in general.
First, according to the analysis of the internal consistency index (Cronbach alpha) and the item-level, CFA fit indexes of PID-5 facets, six scales of the Hungarian PID-5 were poorly fitting based on several fit indices in both samples. These are Emotional Lability, Unusual Beliefs, Depressivity, Perseverance, Risk Taking, and Restricted Affectivity. Some of the fit indices of Emotional Lability and Perseverance did not meet minimal standards in other studies either [18]. Restricted Affectivity and Risk Taking also showed inadequate model fit in another research analyzing the psychometric properties of another translation of the Hungarian PID-5 [3,19]. In our research, the only scale with an unacceptable Cronbach alpha level was Suspiciousness (.54) in the non-clinical sample. The reliability of the Suspiciousness scale was weak in several other psychometric analysis of PID-5 [3], suggesting that this is not a specific characteristic of the Hungarian version of PID-5. We analyzed the factor loadings of the items of the 6 problematic scales and some of them were very low, under .2 (Emotional Lability: item 18, 138, 181, Unusual Beliefs: item 143, Depressivity: item 86, Perseverance: item 46, Restricted Affectivity: item 45, Risk taking: items 98, 164). It might be necessary to revise and rewrite these problematic items in order to increase reliability.
Second, temporal consistency of the Hungarian PID-5 has been confirmed by a one-month test-retest reliability analysis. Based on Cohen's d values, small to no change was observed along with PID-5 facet scales within the one-month interval. The small amount of change observed in the Negative Affectivity (d = -.30), and in the Disinhibition (d = -.20) domains is similar to the findings of Wright and colleagues [8] who observed a small degree change in 6 out of the 25 traits after 1,5 years. Wright considers that the small negative change might be explained by some amelioration of symptomatology and shows that although PD traits are highly stable, some traits might be dynamically associated with the change of psychosocial functioning over time. In our study, test-retest correlation coefficients were between .53 (Callousness) and .86 (Risk-taking), which reflects a moderate to strong connection between the two measurements. In comparison with other self-report measures of psychopathology in other studies [20], PID-5 scales have similar stability coefficients.
Third, we did not find significant differences among the three different models in terms of the factor structure of PID-5 traits. According to our analysis, those models that include all the 25 traits fit weakly in every domain based on the CFI scores. However, the APA model-that builds up the five domains out of 15 facets-, was found to fit slightly better than the others. The Disinhibition domain proved to be a good fit in both samples, and the Detachment domain was fitting well in the clinical sample based on every fit index. The weak CFI scores could be explained by that many items of the scales load onto several factors simultaneously, however, this does not mean that the scales are inconsistent because they have acceptable Cronbach alpha levels.
Fourth, validity of the PID-5 instrument is acceptable in the clinical and the non-clinical sample as well. PID-5 scales are significantly correlated with SCID-II total PD score and SCL-90-R global severity index. Furthermore, different personality disorder scores of SCID-II-PQ are significantly correlated with the expected PID-5 domains and traits with moderate or strong effect size. Moreover, PID-5 domains show significant correlations with all the SCL90-R scales, and as expected we found strong relations between the PID-5 facets and their SCL-90-R counterparts. These results support the hypotheses that the Hungarian PID-5 is a valid measurement of personality pathology.
However we found some unexpected results as well. PID-5 Intimacy avoidance shows only a weak, almost negligible relationship with Avoidant PD, and has no significant correlations with 7 out of the 9 SCL-90-R dimensions, but only in the clinical sample. According to these results, further studies are needed to understand how PID-5 Intimacy avoidance scale can describe the pathological avoidance behavior. We also found that PID-5 Risk taking scale does not correlate significantly with Global Severity Index of SCL-90-R, and 8 of the 9 SCL-90-R subscales in the clinical sample, and although its relationship is significant with Borderline PD, it is only with weak effect size. Furthermore, Risk Taking is one of the four PID-5 traits that does not differentiate significantly between clinical and non-clinical samples. Our results support that Risk taking is a reliable scale of PID-5, but the question arises as to how pathological it can be considered. There are two other PID-5 traits, Deceitfulness and Manipulativeness, which are not correlated significantly with many SCL-90-R dimensions. It suggests that these traits might be independent from psychological distress.
Fifth, we found significant differences among clinical and non-clinical participants in four domains (no difference in Antagonism) and in 22 out of 25 facets, with mostly medium effect sizes. (See Table 8.) In the majority of comparisons, clinical participants received significantly higher scores than non-clinical participants. No significant differences were observed for the facets of Deceitfulness, Manipulativeness and Risk Taking between the two samples; these facets all belong to the Antagonism domain. This finding is only partially in line with Bach and colleagues' findings [21], where they observed no significant differences between clinical and non-clinical groups for the facets of Manipulativeness and Attention Seeking. Scores in the Antagonism domain are relatively low in both samples, meaning that antisocial personality functioning was not characteristic in our clinical sample either. The low level of Antagonism domain in the clinical sample might be explained by the fact that our clinical sample consisted of the participants of the four weeks long psychotherapy program of our department and severe antisocial traits and the diagnosis of antisocial personality disorder with traits of primary psychopathy are exclusion criteria for our treatment program.
Sixth, we found significant differences between the profiles of females and males in PID-5 domains and facets in both samples. Females presented higher scores in the Negative Affectivity domain in both samples, while men score significantly higher in the Antagonism domain in both samples. However, based on effect sizes, the difference between genders in Negative Affectivity is small. In this domain, Emotional Lability scores are higher in females and the difference has medium effect size, but with Anxiousness and Separation Anxiety differences are negligible.
These results are partially in line with the findings of Bastiaens et al. [22] on a clinical sample, where females scored higher on Negative Affectivity and lower on Antagonism than males. However, Bastiaens and colleagues found that females scored higher on Disinhibition, which domain showed no difference in neither of our samples. Moreover, in our non-clinical sample at the facet level males scored higher on the Irresponsibility, Distractibility, Manipulativeness and Risk Taking scales, which is in accordance with the results of Fossati et al. [23], and with reports of higher scores on Conscientiousness in females [24,25].
Seventh, regarding age, a significant negative relation was found with Negative Affectivity, Antagonism, Disinhibition, and Psychoticism in the non-clinical sample. In the clinical sample, only Antagonism and Disinhibition showed significant negative relation with age. In previous PID-5 studies, inverted relations were found between age and the facets of Antagonism, Disinhibition and Psychoticism [23]. In a different study, similar associations were found in a clinical sample [22]. In a non-clinical sample, again, an inverted relation was found between age and Antagonism and Disinhibition [26]. Our results are in line with the results of longitudinal studies in the US and cross-sectional studies from many cultures that were using self-report measures of personality traits, namely that the broad factors of Neuroticism, Extraversion, and Openness to Experience decline from adolescence to adulthood, whereas Agreeableness and Conscientiousness increase [27].
Eighth, we did not find strong significant associations between education and PID-5 traits and domains. This is contradicted by some earlier findings, for example the study of Bastiaens et al. [22] found that Disinhibition was significantly negatively related to education.
Our results are limited by that our measure of personality disorders (SCID-II-PQ) was based only on self-reports. We did not measure the symptoms of Antisocial personality disorder since our ethical committee did not allow us to use the antisocial modul of the SCID-II-PQ for legal and ethical reasons. There was many data missing in the clinical sample, which led to a smaller sample size than originally planned.
Even in light of these limitations, our study shed new light on some aspects of PID5, as well as replicating some previous results [12,28] in a linguistically and culturally different non-clinical and clinical sample.